Journal of Experimental Psychology: Human Perception and Performance

During visual search, we rely on both selective attention and spatial predictions to guide our behavior. However, whether and how these mechanisms interact is largely unclear. Using a contextual cueing paradigm, we investigated whether learning and exploitation of spatial predictive context can occur outside attentional focus. Repeating search scenes enabled distractor context to serve as a contextual cue predicting target location. Participants searched for a target among distractors in two colors: the same as the target-to-be-searched (attended context) or a different color (ignored context). Halfway through the experiments, we changed the target color, thereby altering the attentional status of distractor contexts while maintaining their spatial predictiveness. In Experiment 1, where participants regularly switched between target colors, we found exploitation of spatial predictive context both within and outside attentional focus. However, in Experiment 2, where attention was more stably focused on one target color, only the attended predictive context was exploited before transfer. Intriguingly, after transfer, previously ignored predictive context showed immediate benefits, revealing latent learning. These findings demonstrate a dynamic competition between selective attention and spatial predictions: while learning occurs independently of attention, exploitation may require attentional selection. Our results suggest that selective attention gates the influence of spatial predictions on behavior, with gating strength determined by the stability of attentional control. Significance statementThis research addresses a fundamental question in cognitive science: can we learn from visual information we are not paying attention to? Our findings reveal a dynamic competition between selective attention and spatial prediction that can explain conflicting findings in the literature. We demonstrate that specifically the ability to exploit predictive spatial context depends critically on attentional stability. When required to flexibly switch attentional sets, we can benefit from both attended and unattended predictive information. However, when attention remains focused, unattended information is learned, but only becomes accessible when it is part of our attentional focus. These results reveal that the filtering operation of selective attention and the utilization of spatial prediction during visual search may actually compete dynamically, with selective attention ultimately constraining predictions influence on behavior. This advances our understanding of how these fundamental cognitive mechanisms interact under competition.

In visual search, improved performance when a target appears at a recently cued location is taken as strong evidence that attention was shifted to this cue. Here, we provide evidence challenging the canonical interpretation of spatial-cueing (or cue-validity) effects and supporting the Priority Accumulation Framework (PAF). According to PAF, attentional priority accumulates over time at each location until the search context triggers selection of the highest-priority location. Spatial-cueing effects reflect how long it takes to resolve the competition and can thus be observed even when attention was never shifted to the cue. Here, we used a spatial-cueing paradigm with abruptly onset cues and search displays varying in target-distractor similarity. We show search performance on valid-cue trials deteriorated the more difficult the search, a finding that is incompatible with the standard interpretation of spatial-cueing effects. By using brief displays (Experiment 1) and by examining the effect of search difficulty on the fastest trials (Experiment 2), we invalidate alternative accounts invoking post-perceptual verification processes (Experiment 1) or occasional failures of the onset cue to capture attention (Experiment 2). In Experiment 3, we used a combination of the spatial-cueing and dot-probe paradigms. We show that the events that occurred in both the cue and search displays affected attentional distribution, and that the relative attentional priority weight that accumulated at the target location determined how easily the competition was resolved. These findings fully support PAFs predictions. Public significance statementMany studies aim at establishing whether certain objects mandatorily capture our attention. Here, we show that there is no "yes-or-no" answer to this question because the context in which an object appears determines whether this object captures attention. We show that our attention is not shifted to the highest-priority object at any given time: instead, information about priority is collected across time until some signal indicates that the appropriate moment for deploying our attention has arrived. Striking failures to notice conspicuous events routinely illustrate how limited our attentional system is: we can attend to very few objects at any given time, and probably to just one. In natural conditions, when we move the focus of our attention from one object to another, we also shift our gaze towards the attended location: this allows us to place the object of most interest in the center of our fovea, which maximizes the quality of its perceptual processing. Tracking the locus of such overt attention is easily achieved by using eye-tracking devices. However, in order to isolate the benefits of attention from the benefits of visual acuity, one must study covert attention - that is, attentional shifts in the absence of eye movements. These shifts are not directly observable and must therefore be inferred using indirect measures of processing.

Saccades made during memory maintenance prioritize memory for the saccade target, but it is unclear if this benefit is specific to a location or extends across memorized objects. In three experiments, we examined whether saccadic selection spreads to other locations within the same object. In Experiment 1, we asked observers to remember three oriented Gabors presented either within contour-defined objects or without object structure. A subsequent movement cue prompted observers to move their eyes to the indicated location. We then probed memory for stimuli at locations equidistant from the saccade target, in either the same or a different object. Memory was best for stimuli at locations congruent with the saccade target, and consistently weaker for other stimuli presented in the same or a different object than the saccade target. In Experiment 2, we created more complex objects by adding more object features to the stimulus. Again, memory performance was best for stimuli congruent with the saccade target location, whereas memory in incongruent trials was worse and similar for stimuli in the same and different object as the saccade target. In Experiment 3, we tested if saccadic selection is present and propagates within the object in a change detection task. Again, memory performance (i.e., change detection) was best at the saccade target location. However, this memory benefit also spread to other locations within the same object. Our results imply that saccadic selection in visual working memory is primarily space-based but can also spread towards locations within the object where a saccade was directed.

Static statistical regularities in the placement of targets and salient distractors within the search display can be learned and used to optimize attentional guidance. Whether statistical learning also extends to dynamic regularities governing the placement of targets and distractors on successive trials has been less investigated. Here, we applied the same dynamic cross-trial regularity (one-step shift of the critical item in clock-/counterclockwise direction) either to the target or a distractor, and additionally varied whether the distractor was defined in a different (color) or the same dimension (shape) as the target. We found robust learning of the predicted target location: processing of the target at this (vs. a random) location was facilitated. But we found no evidence of proactive suppression of the predictable distractor location. Facilitation of the anticipated target location was associated with explicit awareness of the dynamic regularity, whereas participants showed no awareness of the distractor regularity. We propose that this asymmetry arises because, owing to the targets central role in the task set, its location is explicitly encoded in working memory, enabling the learning of dynamic regularities. In contrast, the distractor is not explicitly encoded; so, statistical learning of distractor locations is limited to static regularities. Public significance statementCan we learn the cross-trial dynamic regularity of a target or a task-irrelevant salient distractor (e.g., one-step shift of the critical item in clock-/counterclockwise direction) to boost search performance? The present study found robust learning of the predicted target location, but no evidence of proactive suppression of the predictable distractor location. Facilitation of the anticipated target location was associated with explicit awareness of the dynamic regularity. This asymmetry highlights the important role of the target-centered task set in the learning of dynamic regularities.

Visual working memory allows for the brief maintenance of information to serve behavioral goals. It has been shown that when the specific action required to serve a future goal is predictable, people can flexibly change a visual memory representation to incorporate an action-based one, demonstrating the goal-oriented nature of visual working memory. Can such flexibility also be observed within the visual domain, between color and space? In this eye-tracking study, participants remembered either a centrally presented color or a spatial position around fixation. Critically, when remembering a color the response wheel was either randomly rotated, or shown at a fixed rotation, on every trial. When fixed, every target color could be associated with a predictable position on the wheel during response. Do people incorporate this added spatial information in their behavior? Participants utilized color-space associations when remembering color: Response initiation happened faster when the color wheel was fixed compared to random, irrespective of whether an action could be planned or not. Next, we showed that gaze was biased towards the position of the spatial memory target during the delay, extending previous work on gaze biases. Importantly, also when remembering a color, gaze was biased towards the anticipated position of that color on the response wheel when it was fixed. Together, our results show a behavioral benefit of added spatial information for color memory, and systematic changes in gaze that reflect flexible utilization of space.

When people try to remember visual information, they often move their eyes similarly to encoding. The mechanism underlying this behavior has not yet been fully understood. Specifically, it is unclear whether the purpose of this behavior is to recreate the visual input produced during encoding, or the motor and spatial elements of encoding. In this experiment, participants (N=40) encoded pairs of greyscale objects, overlaying colored squares. During test, participants were asked about objects orientation, while presented with squares of the same colors, either at the same location (controlled trials) or switched in their locations (test trials) relative to encoding. Results show that during test trials, participants tended to gaze at the square appearing at the location where the remembered object was previously presented, rather than on the square of the same color. This indicates a superiority of motor and spatial elements of eye movements rather than near-peripheral visual cues.

Attention is automatically guided towards stimuli that match the contents of working memory. This has been studied extensively using simplified computer tasks, but it has never been investigated whether (yet often assumed that) memory-driven guidance also affects real-life search. Here we tested this open question in a naturalistic environment that closely resembles real life. In two experiments, participants wore a mobile eye-tracker, and memorized a color, prior to a search task in which they looked for a target word among book covers on a bookshelf. The memory color was irrelevant to the search task. Nevertheless, we found that participants gaze was strongly guided towards book covers that matched the memory color. Crucially, this memory-driven guidance was evident from the very start of the search period. These findings support that attention is guided towards working-memory content in real-world search, and that this is fast and therefore likely reflecting an automatic process. Significance statementA core concept in the field of visual working memory (VWM) is that visual attention is automatically guided towards things that resemble the content of VWM. For example, if you hold the color red in VWM, your attention and gaze would automatically be drawn towards red things in the environment. So far, studies on such memory-driven guidance have only been done with well-controlled computer tasks that used simplified search displays. Here we address the crucial and open question of whether attention is guided by the content of VWM in a naturalistic environment that closely resembles real life. To do so, we conducted two experiments with mobile eye tracking. Crucially, we found strong memory-driven guidance from the very early phase of the search, reflecting that this is a fast, and therefore likely automatic, process that also driven visual search in real life.

Attention optimizes learning by filtering relevant information to build conceptual knowledge. However, how learned concepts, once encoded in memory, subsequently guide attentional processes remains an intriguing question. We propose that concept learning leads to the emergence of attentional templates that store goal-relevant representations, thereby actively guiding attention allocation. Participants completed two separate learning tasks and a test, wherein each trial began with a cue, indicating which learning task should be employed. Random test trials included a probe instead of concept specific features: a small arrow appeared at a feature location that was relevant (i.e., valid) or irrelevant (i.e., invalid) for the cued task. Successful learners were faster at responding to valid probes than invalid, demonstrating the deployment of concept-specific attentional templates. Importantly, the efficiency of this attention allocation was tied to concept learning success, with higher learning performance yielding greater response time benefits at test. Thus, our results reveal that learning builds behaviourally relevant attentional templates, and subsequently, learned concepts in memory guide attention by deploying these templates, a phenomenon that we introduce as learning-guided attention. This work provides novel insights into the dynamic interplay between learning, memory, and attention. Significance StatementExtensive work shows that attention selects the most relevant information while learning new knowledge. Theoretically, the interaction between learning and attention is bidirectional; learned knowledge, in turn, guides attention to relevant information for that context. However, the mechanism by which knowledge in memory directs attention has remained largely unexplored. By developing an experimental paradigm that bridges the learning and attention literatures, we demonstrate that learning builds "attentional templates" which capture what is relevant in a specific learning context. While utilizing learned knowledge, individuals deploy these templates to allocate their attention to the most relevant information for a given situation. We introduce this phenomenon as learning-guided attention, providing novel insights into the dynamic interplay between learning, memory, and attention.

The decoy effect occurs when adding an inferior third option biases choice between two others, even though the decoy is rarely chosen. While robust in value-based decisions, evidence in perceptual tasks is mixed. Using the rhesus-macaque paradigm from Parrish et al. (2015), we tested whether a perceptual decoy effect generalizes to humans. Participants (n = 50) completed 400 trials. Contrary to our preregistered prediction, we found no reliable decoy effect. Accuracy improved on the hardest trials (Level 1) when a decoy was present, response times were slower in decoy conditions than baseline, and accuracy was higher for tall versus wide rectangles, consistent with the vertical-horizontal asymmetry. The relatively wide spacing of stimuli may have reduced grouping and attentional clustering; because spacing was not manipulated, this remains a hypothesis for future tests. Results suggest that context effects in perceptual choice operate under narrower boundary conditions than in value-based domains.

The ability to store information in visual working memory is essential to plan and successfully execute memory-guided actions in natural human behavior. Typically, visual working memory research investigates how storing affects subsequent action. In doing so, however, the importance of how the action affects prior storing remains underappreciated. Therefore, we here question how the required precision for an action to succeed, affects how relevant visual information is encoded, maintained and finally acted on. To this end, we had participant memorize 1, 2 or 4 colors for delayed continuous report. Crucially, we manipulated how (im)precise the report was allowed to still be marked correct. Behavioral results showed that for actions with higher required precision, reports became more precise, but only when one or two colors were memorized. Also, reports became slower with higher required precision, regardless of the number of colors. By leveraging pupillometry, we further showed that with higher required precision, 1) colors were encoded deeper (since pupils constricted more during presentation), and 2) more effort was exerted to maintain the colors (since pupils dilated more during retention). Moreover, we found that participants kept exerting more effort to be precise (with increasing precision requirements), even when additional effort did not result in better performance anymore. Our findings demonstrate that humans consider their intended actions when encoding and maintaining information in visual working memory. Our results highlight the essential role of action in understanding how visual information is stored during natural goal-directed behavior. 242 words

When attempting to recall previously seen visual information, people often move their eyes to the same locations where they initially viewed it. These eye-movements are thought to serve a role in enhancing memory retrieval, although the exact mechanism underlying this effect is yet unknown. To investigate this link between eye-movements and memory, we conducted an experiment with 80 adult participants. Participants were asked to perform a memory retrieval task, while viewing either the same visual context as during encoding or an altered one. Results showed that the benefit of eye movements to memory retrieval was dependent on the visual input. This suggests that the contribution of eye-movements to memory may not be from the motor behavior itself, but from its visual consequences. Our findings thus challenge the hypothesis that eye movements act as a motor retrieval cue and support the view that their visual consequences act as a sensory one. Statement of RelevanceAn intriguing question in cognition is how humans encode memorized material and what helps them retrieve it. It is known that when an action or stimulus is repeated both when information is encoded and when it is retrieved, this can act as a retrieval cue and enhance memory performance. It is also known that people tend to reenact the same eye movements during retrieval as they did during encoding, and this behavior is associated with higher memory performance. This has led to the hypothesis that eye movements act as a retrieval cue. However, we challenge this hypothesis by showing that the visual consequences of eye movements, rather than the motor action that accompanies them, is the key factor for memory enhancement. Understanding the factors that influence memory provides crucial insight into the relationship between external behaviors and internal memory processes, leading to significant implications for the educational and clinical settings.

How does the structure of events influence the when and the where of experience in comparison to the what? We developed a novel virtual reality (VR) environment to understand how the quantity of information within nested structures influence participants memory for events. Participants moved through a series of virtual rooms (events) where images (items) appeared in randomised locations on a 3 by 3 grid located on a wall. Participants were asked to remember the what (old/new), when (timeline location), and where (grid location), of the images they experienced. Two types of nested events were tested (6 rooms, each containing 4 images; 3 rooms, each containing 8 images) without a difference in the number of seconds of presentation. We found a strong temporal compression effect at nested levels in which participants remembered early items and events happening later, and later items and events happening earlier, than the original experience. Crucially, presenting four-item events resulted in a greater compression rate than eight-item events. We also found greater temporal distances between pairs of items occurring within eight-item events than pairs of items which occurred on either side of a boundary. Memory for when depends on the compression of information within events.

Visual working memory (VWM) is traditionally studied while constraining eye movements and limiting access to visual input, yet in natural vision humans constantly explore and resample their environment. Only a few studies have examined VWM utilization when participants were allowed to interact with the environment and found that participants often preferred to resample their environment rather than rely on VWM storage. However, since eye movements were not controlled in these studies, the link between VWM utilization and free visual exploration remained unknown. In two experiments (N = 40), we investigated how visual exploration shapes reliance on VWM versus perceptual input. Participants searched for a common target across two item sets and could either store multiple items for comparison or repeatedly resample the sets by switching between them. Results revealed that when switching was achieved through eye movements, participants consistently relied more on visual resampling and less on VWM; in contrast, when switching required a manual response, they shifted toward greater VWM use. This pattern persisted even when peripheral input was equated, suggesting that natural exploration through eye movements reduces the cognitive cost of acquiring visual information, leading to a strategic reduction in VWM use. Our findings challenge fixation-based approaches to VWM research and highlight the importance of studying cognition under ecological viewing conditions.

Visual working memory (vWM) is often linked to conscious experience and visual imagery, but it is typically described as a system that stores separate, independent items. These assumptions are difficult to reconcile, given the unified nature of conscious experience. Here, we test the hypothesis that vWM relies on at least two distinct representations: an underlying, unconscious memory trace and a consciously accessible, integrated representation. A total of 216 participants performed a change-detection task, in which they rated their perceptual awareness of the memory display during the maintenance interval. Critically, we manipulated the statistical properties of the displays (average item size and size variability) to probe sensitivity to unified ensemble-level structure. Results revealed a dissociation between subjective and objective measures. Perceptual awareness increased for displays with larger, more variable items, whereas objective performance improved for displays with smaller, less variable items. Despite this difference, subjective awareness still predicted performance, and even incorrect responses showed consistent biases rather than random guesses. Importantly, individual differences in imagery vividness (VVIQ) were selectively associated with subjective awareness and estimation bias, but not with objective correctness. These precision biases were further shaped by display statistics, suggesting that multiple representations can guide behavior. Together, our findings support a reinterpretation of vWM performance in which task responses can draw on both unconscious and consciously accessible representations. One possible explanation for these behavioral patterns is that subjective experience reflects integrated, ensemble-like representations, while objective performance depends more strongly on item-specific information. Public significance statementsWorking memory allows us to temporarily hold and use information, and differences in this ability are closely linked to broader cognitive skills such as intelligence. This study shows that these differences may not depend only on how much information people can store, but also on how they experience it: some individuals appear to rely more on consciously accessible, image-like representations, especially when memory is uncertain or prone to error. By demonstrating that subjective experience and the vividness of imagery can shape behavior independently of objective accuracy, these findings suggest that how we use memory may be as important as how much we can store, with implications for understanding individual differences in cognition.

Results from psychophysical studies using visual adaptation suggest that launch detectors in the visual system underlie the perception of causality in simple visual events. These detectors respond to events in which one stimulus collides with another stimulus (i.e., a launch), and do not respond to events where one stimulus passes over another (i.e., a pass). Prolonged visual adaptation to launches significantly reduces observers propensity to see causal launches at the same retinotopic location. This finding could be taken to indicate that launch detectors are necessary for the local detection of causal launches. However, contextual events--that are spatially separated from the test event location--shift observers perception of a causal relation in the direction of the type of contextual event (Scholl & Nakayama, 2002), providing evidence for spatial integration beyond a specific retinotopic location. Here, we used visual adaptation as a tool to investigate whether the contextual influence on causal perception relies on local launch detectors. Before and after adaptation, we determined the proportion of reported launches in ambiguous test events in the presence of no context, launch context, and pass context events. We hypothesized that if the contextual influence relies on (unadapted) local launch detectors, then visual adaptation should affect the contextual influence on causal perception. Before adaptation, a launch-context event increased the proportion of reported launches (while a pass context event decreased it). Visual adaptation to launches significantly decreased the proportion of reported launches in no-context trials, but did not affect perceptual reports in no-context trials. In fact, contextual influences, expressed relative to no-context trials, emerged strongly after adaptation. This result suggests that context effects override strong negative aftereffects from adaptation, indicating that contextual influences operate at a level that bypasses the local launch detector at the adapted location.

Viewpoint effects on object recognition interact with object-scene consistency effects. While recognition of objects seen from "accidental" viewpoints (e.g., a cup from below) is typically impeded compared to processing of objects seen from canonical viewpoints (e.g., the string-side of a guitar), this effect is reduced by meaningful scene context information. In the present study we investigated if these findings established by using photographic images, generalise to 3D models of objects. Using 3D models further allowed us to probe a broad range of viewpoints and empirically establish accidental and canonical viewpoints. In Experiment 1, we presented 3D models of objects from six different viewpoints (0{degrees}, 60{degrees}, 120{degrees}, 180{degrees} 240{degrees}, 300{degrees}) in colour (1a) and grayscaled (1b) in a sequential matching task. Viewpoint had a significant effect on accuracy and response times. Based on the performance in Experiments 1a and 1b, we determined canonical (0{degrees}-rotation) and non-canonical (120{degrees}-rotation) viewpoints for the stimuli. In Experiment 2, participants again performed a sequential matching task, however now the objects were paired with scene backgrounds which could be either consistent (e.g., a cup in the kitchen) or inconsistent (e.g., a guitar in the bathroom) to the object. Viewpoint interacted significantly with scene consistency in that object recognition was less affected by viewpoint when consistent scene information was provided, compared to inconsistent information. Our results show that viewpoint-dependence and scene context effects generalize to depth rotated 3D objects. This supports the important role object-scene processing plays for object constancy.

Recent research indicates there is overlap in the neural resources used during imagery and visual short-term memory. But do visual short-term memory and visual imagery operate on similar representations during recall? Here we investigated this question by asking participants to perform a delayed match to sample task for the contrast of visual gratings as cues. In the "Imagery" condition, participants were asked to form an accurate mental image of the visual cue, and at the end of the trial, perform a matching task on the mental image contrast. In the "Memory" condition, participants were not required to perform visual imagery but merely instructed to perform the delayed contrast-matching task. In Experiment 1, participants were told at the beginning of each block whether to engage in memory or imagery. The results showed that for the relevant contrast feature, matching judgments were more accurately in the "Memory" than in the "Imagery" condition. Thus imagery did not maintain an accurate representation of the encoded image, even when the visual features could still be maintained in visual short-term memory. In Experiment 2, participants were required to engage in memory and imagery simultaneously, and were told which to base their judgment on after each trial. The key finding was that the superior accuracy for memory over imagery remained, indicating that the contents of VSTM and imagery are based on distinct representations.

Humans can extract regularities from their environment, enabling them to recognize and predict sequences of events. The process of regularity extraction is called statistical learning and is generally thought to occur rapidly and automatically; that is, regularities are extracted from repeated stimulus presentations, without intent or awareness, as long as the stimuli are attended. We hypothesized that visual statistical learning is not entirely automatic, even when stimuli are attended, and that the learning depends on the extent to which viewers process the relationships between stimuli. To test this, we measured statistical learning performance across seven conditions in which participants (N=774) viewed image sequences. As task instructions across conditions increasingly required participants to attend to relationships between stimuli, their learning performance increased from chance to robust levels. We conclude that the learning observed in visual statistical learning paradigms is, for the most part, not automatic and requires more than passively attending to stimuli.

Human visual experience of objects comprises a combination of different visual features, such as their color, position, and shape. Spatial attention is thought to play a role in creating a coherent perceptual experience, integrating visual information coming from a given location, but the mechanisms underlying this process are not entirely understood. Deficits of spatial attention in which this integration process does not occur normally, such as neglect, can provide insights regarding the mechanisms of spatial attention in visual object recognition. In this study, we describe a series of experiments conducted with an individual with neglect, XX. XX presents characteristic lack of awareness of the left side of individual objects, evidenced by poor object and face recognition, and impaired word reading. However, he exhibits intact recognition of color within the boundaries of the same objects he fails to recognize. Furthermore, he can also report the orientation and location of a colored region on the neglected left side despite lack of awareness of the shape of the region. To our knowledge, selective lack of awareness of shape despite intact processing of basic visual features in the same spatial location has not been reported previously. XXs performance raises intriguing questions and challenges about the role of spatial attention in the formation of coherent object percepts and visual awareness.

When we ask people to hold a color in working memory, what do they store? Do they remember colors as point estimates (e.g. a particular shade of red) or are memory representations richer, such as uncertainty distributions over feature space? We developed a novel paradigm (a betting game) to measure the nature of working memory representations. Participants were shown a set of colored circles and, after a brief memory delay, asked about one of the objects. Rather than reporting a single color, participants placed multiple bets to create distributions in color space. The dispersion of bets was correlated with performance, indicating that participants internal uncertainty guided bet placement. Furthermore, relative to the first response, memory performance improved when averaging across multiple bets, showing that memories contain more information than can be conveyed in a single response. Finally, information about the item in memory was present in subsequent responses even when the first response would generally be classified as a guess or report of an incorrect item, suggesting that such failures are not all-or-none. Thus, memory representations are more than noisy point estimates; they are surprisingly rich and probabilistic.